On Feature Subset Selection for Fuzzy and Classic Machine Learning Classification Methods

Feature subset selection supports the classification task by reducing the search space as well as by removing irrelevant and random features, which might compromise the resulting classification model. Decision trees perform an embedded feature selection as they select only the relevant features for the splitting of the datasets during the induction process. FUZZYDT is a fuzzy decision tree which uses entropy and information gain in its induction process. Its main advantage over classic decision tree algorithms is the transformation of the attributes into fuzzy linguistic attributes, adding interpretability to the induced models and allowing the processing of imprecision and uncertainty through the use of the fuzzy set and fuzzy logic theories. Filters are also widely used as they present low computational cost and can be applied as a preprocessing step. The large differences in the available approaches for feature selection motivated us to empirically test some methods specifically for fuzzy classification systems. Our initial hypothesis was that FUZZYDT would present better results for fuzzy classification systems when compared to other methods due to the fact that such fuzzy systems and FUZZYDT share the definition of the attributes in terms of fuzzy sets. The experiments carried out showed that the CFS filter produced better results than other filters, C4.5, and FUZZYDT. Such results, although contrary to our hypothesis, are relevant for our research with fuzzy systems, especially for genetic fuzzy systems due to their high computational cost, as CFS is simple and presents low computational cost. Keywords—Feature subset selection, fuzzy logic, fuzzy classification systems, decision trees.